MetaQuerier over the Deep Web: Shallow Integration across Holistic Sources

The Web has been rapidly “deepened” by myriad searchable databases online. To enable effective access to the “deep Web,” we are building the MetaQuerier– for exploring and integrating databases on the Web. Such metaquerying must tackle integration at a large scale (as sources are proliferating online) and of a dynamic nature (as each query will access different sources). Toward such integration, our approach hinges on the insight that the challenge of large scale is itself an opportunity: We observe that the desired “semantics” often connects to surface presentation characteristics, through some hidden regularities over many sources. Generalizing our recent works, this paper thus proposes our approach of shallow integration across holistic sources – to discover desired semantics by exploiting the hidden regularities of shallow clues across many sources holistically. As evidences, we have studied two concrete problems: 1) query-interface understanding based on hidden syntax, and 2) query-interface matching based on hidden statistic models. Our experience indicates high promise for employing shallow techniques across holistic sources.